Ceph support We design, migrate and run the storage under your Proxmox VE, OpenStack and Kubernetes clusters.
We get called into Ceph for three things, mostly: standing up the cluster properly the first time, rescuing the one someone else half-built, or replacing a SAN array that now costs more than the rest of the kit. Covers RBD, CephFS and RGW (S3) on OpenStack, Kubernetes or Proxmox VE.
What Ceph is
and when it fits
Ceph runs on Linux and standard x86 hardware, and one cluster gives you block (RBD) for virtual machines, POSIX files (CephFS) for whatever gets mounted from a hundred places at once, and S3 objects (RGW) for backup and application state. Data is replicated across servers; a dead disk, a dead host, or even a dead room doesn't take production down.
It's what holds up the private cloud when you decide not to depend on a hyperscaler, and it makes sense when your SAN array is full and the next one costs more than a car. It scales by adding servers, not by buying a new box from the same vendor.
Certified SAP HANA, no. Databases with strict sub-millisecond latency, no either. There Ceph is not the answer and we'll spare you the project. For almost everything else in the data centre — virtualisation, private cloud, backup, internal S3, containers — it fits.
How to learn Ceph from scratch →
Official project · ceph.io ↗ · Technical documentation ↗
What we cover in Ceph
The full life of the cluster with the same team: design, migration off ageing arrays, tuning when it goes slow, and the 3am shift when something falls over.
Ceph RBD
The disk your VMs actually run on, served by Ceph instead of a SAN. Snapshots and clones in seconds because they're copy-on-write, not full copies. Talks natively to Cinder, libvirt/KVM and Proxmox VE.
CephFS
A POSIX file system that gets mounted from a hundred machines at once without a single metadata server becoming the bottleneck. When someone asks for "NFS, but properly", this is what we deploy. How to run it in HA with Ganesha →
Ceph RGW · Object storage
An S3 endpoint pointing at your own cluster, not at AWS. Bucket policies, IAM, versioning, lifecycle. Veeam and IBM Storage Defender treat it as any other bucket.
S3 · Swift · Veeam · IBM Defender Migrating from IBM COS to Ceph object storage →Rook / Ceph CSI
Pods need volumes that survive a reschedule. With Rook Ceph lives inside the K8s cluster itself; with Ceph CSI it connects to an external one. Both give you real snapshots and RWX volumes.
RWO · RWX · CSI snapshotsCeph on Proxmox VE
Three nodes with compute and storage on the same box, without buying vSAN and without a per-CPU licence. Proxmox integrates Ceph natively; we size it and tune it so it holds production, not just demos.
vSAN alternativeCeph with NVMe for AI, HPC and analytics
A GPU-hour costs more than a month of an OSD; storage can't be the bottleneck. All-NVMe pools, 100 GbE or RDMA networking, and a CRUSH map designed so replicas don't fight for the same switch.
Ceph on Proxmox VE
Proxmox ships Ceph built in, so the storage gets decided when the cluster is built rather than afterwards. Four things change the outcome, and three of them cost money if you find them late.
ZFS or Ceph?
It depends on whether the VMs can stop. ZFS replication is asynchronous and runs between pairs of nodes: cheaper and simpler to operate, and in a failure you lose whatever was written since the last replication run — as recent as the schedule you set. Ceph is synchronous and shared: the data is in the cluster before the write is acknowledged, so any node starts any VM without moving anything. Two nodes and no high-availability requirement, ZFS is plenty. Three or more with services that cannot go down, Ceph.
ZFS async · Ceph syncThe network is where it breaks
On the default replicated pool every client write generates two more copies that travel across the network, and the rebalance after losing a disk competes for the same cable as your VMs. 1 GbE does not carry that: 10 GbE is the floor, with the cluster network separated from management, and with NVMe OSDs it runs out quickly. Size it up front, because changing the network on a cluster already in service is a separate project.
10 GbE floor · separate cluster networkGetting off vSAN
Coming from vSphere with vSAN, the functional equivalent is Proxmox with Ceph: compute and storage on the same boxes. The licensing is worth stating precisely, because it often gets counted wrong: Proxmox runs without a subscription, and the subscription that exists is sold per socket and buys the enterprise repository rather than unlocking features; vSAN is licensed per core with a per-CPU minimum. Which one adds up depends on your node and core count, so it gets worked out with the inventory in front of you. Then there are the operations, which is where the time goes: different tooling, a different failure model and a different place to read the logs.
vSAN alternativeWhere the backup lives
Replicas cover a dead disk. A deletion or ransomware replicates like everything else, so the backup has to live outside the cluster: Proxmox Backup Server on separate hardware, with deduplication, and with verification scheduled so a restore that does not work is not discovered on the day it is needed.
Proxmox Backup ServerWhat we actually do
In-house engineers who sit with your team and know your cluster inside out.
We start by measuring what the cluster will actually do, not what the brief says. That gives us the number of OSDs, the SSD:HDD ratio, the network topology and the CRUSH rules. We deploy with cephadm and don't call it done until fio and rados benchmarks replicate the numbers we promised.
The old array doesn't get powered down until Ceph has been serving the same data for weeks. We move workloads in waves — first the ones you can restart, then the 3am critical ones — and compare checksums before pulling the cable on the SAN.
Ceph performs as well as you let it perform. When it's slow, it's usually the network, the PG count, or a poorly balanced pool; almost never "Ceph is slow". We measure, change one thing at a time, and hand you the benchmarks so you can rerun them yourself.
Prometheus, Grafana and Ceph Dashboard wired into whatever you already run. We don't page you for every scrub; we do page you when an OSD is heading for the graveyard or when capacity is on track to hit the wall in three months.
This is the one that calls us at 3am. Inconsistent PGs, monitors without quorum,
OSDs that won't start, an upgrade that got stuck halfway. 24/7 on-call, contract
SLA. Golden rule: if the cluster is degraded, don't touch it before talking to us.
The most common error and how to fix it →
Response times
Read-only cluster, loss of monitor quorum, or PGs unreachable.
OSDs down, performance degradation or recovery in progress.
Architecture review, tuning and proactive recommendations.
SLA and penalties defined in the contract.
When Ceph wins,
when it doesn't
No tech fits everything. These are the cases where Ceph delivers, and the ones we'd send elsewhere even if it cost us the project.
| What you need to solve | Ceph | What we'd suggest instead |
|---|---|---|
| Replace a SAN array that now costs more than a full hardware refresh | Fits | — |
| Persistent storage for OpenStack or Kubernetes | Reference pattern | — |
| Internal S3 bucket without going through AWS | RGW solves it | — |
| Hyperconverged cluster on Proxmox or KVM, no licence required to run | Mature combo | — |
| Long-term backup and archive with Veeam or Storage Defender | RGW S3 | — |
| Databases needing strict sub-millisecond latency | Don't force it | Local NVMe + app-level replication |
| SAP HANA certified by SAP | Not certified | IBM FlashSystem or another SAP-certified array |
| Home directories for one building (3-5 TB, one site) | Overkill | Classic NFS or a small array |
Ceph almost never
travels alone
It shows up with OpenStack on top, or with Kubernetes. Often both. The people who know the layer below are the same ones who support the layers above.
Distributed storage: block for VMs, POSIX for files, S3 for the rest. What holds everything above it up.
You are hereA real private cloud, off the hyperscaler. Nova, Neutron, Cinder and Glance on Ceph. Training available while we build the support landing.
OpenStack training →OpenShift, Rancher (k3s), Canonical Kubernetes, Talos. Persistent volumes come from Ceph via Rook or Ceph CSI.
Docker & Kubernetes training →Frequently asked questions about Ceph
Ceph is an open source distributed storage platform that runs on Linux and standard hardware. One cluster provides three interfaces: block (RBD) for VMs and databases, POSIX files (CephFS) for shared workloads and S3-compatible object storage (RGW). Data is replicated across nodes and the cluster tolerates disk, server or even whole-room failures if designed accordingly.
When you want to grow without buying a new array every time the current one fills up, when the per-TB licence cost from your vendor grows faster than the business, or when you need S3 and shared files on top of block without licensing three separate products. If your main workload is certified SAP HANA or requires strict sub-millisecond latency, Ceph is not the answer and we will tell you.
Yes, and it is one of Ceph's most mature integrations. Proxmox VE ships with native Ceph support, which lets you build hyperconverged clusters from three nodes upwards: compute and storage on the same machines, without extra licences or vSAN. At SIXE we design and tune that combination for production, not just for the lab.
It depends on whether the VMs can stop. ZFS replication is asynchronous and runs between pairs of nodes: cheaper and simpler to operate, but in a failure you lose whatever was written since the last replication run; live migration sends only the increment since that run, not the whole disk. Ceph is synchronous and shared: any node can start any VM immediately. Two nodes with no high-availability requirement, ZFS is plenty. Three nodes or more with services that cannot go down, Ceph.
Both. Rook is the operator that runs Ceph inside the cluster itself, useful when you want a self-contained stack. Ceph CSI is the driver that connects Kubernetes to an existing external Ceph cluster — the common choice when Ceph already serves other consumers. Works with OpenShift, Rancher (k3s), Canonical Kubernetes and Talos, and provides ReadWriteOnce and ReadWriteMany volumes with snapshots and clones.
Longhorn is lighter and works well for small edge clusters or dev environments where Ceph is overkill. Ceph is the answer when you also need object storage and shared file systems, when the cluster grows beyond a handful of nodes, or when the same storage serves both Kubernetes and non-Kubernetes workloads (VMs, backup, S3). Simple rule of thumb: single small K8s cluster and only block volumes → Longhorn. Everything else → Ceph.
A useful production cluster starts at three nodes — the minimum to keep monitor quorum and a size-3 replication rule. Below that Ceph runs, but as a lab, not a fault-tolerant system. Six nodes start to allow rack-level failure domains, and from ten upwards you are into serious enterprise topologies.
Yes, it is one of the reasons people call us. Diagnosis of inconsistent PGs, loss of monitor quorum, OSDs that will not start, stuck recovery or halted upgrades. 24/7 on-call with contract-backed P1 SLA.
Write to us and we'll look at it
Tell us what's going on: new cluster, one that won't come back, or a SAN array that's getting out of hand. An engineer replies.
Blog!
Our mission
SIXE ensures that your infrastructures based on IBM®, Red Hat®, Canonical®, and SUSE® technologies remain stable, secure, and optimized for as long as you need them.
At SIXE, we are committed to sustainability. That’s why we help you design, implement, and maintain durable and efficient technological solutions.
